Understanding how the brain encodes external stimuli and how these stimuli can be decoded from the measured brain activities are long-standing and challenging questions in neuroscience. In this paper, we focus on reconstructing the complex image stimuli from fMRI (functional magnetic resonance imaging) signals. Unlike previous works that reconstruct images with single objects or simple shapes, our work aims to reconstruct image stimuli that are rich in semantics, closer to everyday scenes, and can reveal more perspectives. However, data scarcity of fMRI datasets is the main obstacle to applying state-of-the-art deep learning models to this problem. We find that incorporating an additional text modality is beneficial for the reconstruction problem compared to directly translating brain signals to images. Therefore, the modalities involved in our method are: (i) voxel-level fMRI signals, (ii) observed images that trigger the brain signals, and (iii) textual description of the images.

Understanding how the brain encodes external stimuli and how these stimuli can be decoded from the measured brain activities are long-standing and challenging questions in neuroscience. In this paper, we focus on reconstructing the complex image stimuli from fMRI (functional magnetic resonance imaging) signals. Unlike previous works that reconstruct images with single objects or simple shapes, our work aims to reconstruct image stimuli that are rich in semantics, closer to everyday scenes, and can reveal more perspectives. However, data scarcity of fMRI datasets is the main obstacle to applying state-of-the-art deep learning models to this problem. We find that incorporating an additional text modality is beneficial for the reconstruction problem compared to directly translating brain signals to images. Therefore, the modalities involved in our method are: (i) voxel-level fMRI signals, (ii) observed images that trigger the brain signals, and (iii) textual description of the images.

Now, there's concern that neuroscientists might be doing the same by developing technologies capable of "decoding" our thoughts and laying bare the hidden contents of our mind. Though neural decoding has been in development for decades, it broke into popular culture earlier this year, thanks to a slew of high-profile papers. In one, researchers used data from implanted electrodes to reconstruct the Pink Floyd song participants were listening to. In another paper, published in Nature, scientists combined brain scans with AI-powered language generators (like those undergirding ChatGPT and similar tools) to translate brain activity into coherent, continuous sentences. This method didn't require invasive surgery, and yet it was able to reconstruct the meaning of a story from purely imagined, rather than spoken or heard, speech.

Corporate giants like Google, Facebook, and IBM are collectively investing billions of dollars in artificial intelligence (AI), bringing together some of the world's brightest minds who claim that new techniques can create machines that think independently and creatively. So with all this effort, money, and hype, where are the smart robots in our society? Where are the AI assistants, co-workers, and companions? Where are all the cool droids and humanoids that science fiction promised us? In order to build AIs with human-like intelligence--AIs who can interact socially, who are able to work with us to achieve goals, and who are behaviorally and intellectually similar to beloved characters from Star Trek and Star Wars--we must first create one fundamental feature almost entirely missing from their current design.

A pair of wealthy Americans are looking for brain implants on the black market that will allow them to communicate with each other using the power of thought. That's according to presidential candidate Zoltan Istvan who believe that in a few decades, we could all be plugged into an AI'matrix'. 'Eventually, this type of technology will allow us to be connected 24/7 to the internet and on social media,' he told the DailyMail.com. 'This is the beginning of the hive mind, where everyone is interconnected to one another.' A pair of wealthy Americans are looking for brain implants on the black market that will allow them to communicate with each other using the power of thought.